A More Scalable Comparision Shopping Agent

Comparison shopping is a merchant brokering process that provides the information about a specific product from several distributed shopping mall sites. The implementation of a comparison shopping agent is a challenging issue mainly because different on-line shopping malls employ their own way of organizing interface components, processing user queries, and displaying search results on the browser. In most of the previous comparsion shoppers, a human designer manually builds the wrapper for each shopping mall that contains rules for extracting product information. This approach is not scalable since the designer must build a wrapper by hand for each new shopping mall. Scalable comparison shoppers such as ShopBot have been suggested that automatically build the wrapper through learning, but since these systems assume some regularities and strong biases, many real-world shopping malls are unable to be recognized. This paper proposes a more scalable comparison shopping agent. Our system presents a robust and automatic shopping mall learning algorithm and an ontology generation method. The main idea of the proposed algorithm is to determine the position of a product description unit from the HTML source of a search result page by recognizing a repeated pattern of logical line information. This positional information is converted into an extraction rule that becomes the main part of the wrapper. This algorithm is simple but robust because no strong biases are assumed, and consequently the success rate of constructing a correct wrapper is very high. We also suggest a mechanism that generates the ontology from the wellstructured outputs and automatically extends the existing ontology by applying it to unstructured search results.